Robust Sound Source Localization Research Articles

Microphone Pair Training for Robust Sound Source Localization With Diverse Array Configurations

Microphone Pair Training for Robust Sound Source Localization With Diverse Array Configurations

CNN-based Robust Sound Source Localization with SRP-PHAT for the Extreme Edge

Robust sound source localization for environments with noise and reverberation are increasingly exploiting deep neural networks fed with various acoustic features. Yet, state-of-the-art research mainly focuses on optimizing algorithmic accuracy, resulting in huge models preventing edge-device deployment. The edge, however, urges for real-time low-footprint acoustic reasoning for applications such as hearing aids and robot interactions. Hence, we set off from a robust CNN-based model using SRP-PHAT features, Cross3D [ 16 ], to pursue an efficient yet compact model architecture for the extreme edge. For both the SRP feature representation and neural network, we propose respectively our scalable LC-SRP-Edge and Cross3D-Edge algorithms which are optimized towards lower hardware overhead. LC-SRP-Edge halves the complexity and on-chip memory overhead for the sinc interpolation compared to the original LC-SRP [ 19 ]. Over multiple SRP resolution cases, Cross3D-Edge saves 10.32%~73.71% computational complexity and 59.77%~94.66% neural network weights against the Cross3D baseline. In terms of the accuracy-efficiency tradeoff, the most balanced version ( EM ) requires only 127.1 MFLOPS computation, 3.71 MByte/s bandwidth, and 0.821 MByte on-chip memory in total, while still retaining competitiveness in state-of-the-art accuracy comparisons. It achieves 8.59 ms/frame end-to-end latency on a Rasberry Pi 4B, which is 7.26× faster than the corresponding baseline.

TDOA-Based Robust Sound Source Localization With Sparse Regularization in Wireless Acoustic Sensor Networks

Time difference of arrival (TDOA) measurements, which are contaminated by large values of error, known as outliers, would have a significant impact on the accuracy of sound source localization (SSL) in wireless acoustic sensor networks (WASNs). Few techniques are reported in the literature to tackle SSL in WASNs by taking TDOA outliers into consideration. To mitigate the effect of outliers on the accuracy of SSL, we propose outlier-resistant robust sound source localization (RSSL) algorithms based on sparse regularization using an unsynchronized network of microphone arrays. The TDOA errors are divided into two components: a) energy-bounded inliers and b) outliers. Assuming that outliers are sparse in the measurement set, we formulate the RSSL problem as that of minimizing the number of outliers, mathematically, a <inline-formula><tex-math notation="LaTeX">$\ell _{0}$</tex-math></inline-formula> (pseudo)-norm optimization problem with non-convex constraints. Five sub-optimal RSSL solvers are derived, among which the first two solvers are applicable to the scenario involving only outliers while the last three solvers concentrate on the scenario incorporating both outliers and inliers. In common, these solvers exploit a convex approximation technique called Concave Convex Procedure to dispose of the non-convex constraints. Differently, the first solver approximates the original <inline-formula><tex-math notation="LaTeX">$\ell _{0}$</tex-math></inline-formula> (pseudo)-norm cost function with the <inline-formula><tex-math notation="LaTeX">$\ell _{1}$</tex-math></inline-formula> norm while a concave surrogate function is adopted in the second solver to yield a tighter approximation to the <inline-formula><tex-math notation="LaTeX">$\ell _{0}$</tex-math></inline-formula> (pseudo)-norm. Apart from the application of these two approximation techniques, the third and fourth solvers relax the non-convex <inline-formula><tex-math notation="LaTeX">$\ell _{2}$</tex-math></inline-formula> norm constraint with the <inline-formula><tex-math notation="LaTeX">$\ell _{\infty }$</tex-math></inline-formula> norm. The fifth solver is dedicated to the <inline-formula><tex-math notation="LaTeX">$\ell _{1}$</tex-math></inline-formula> norm regularization problem with the Lasso formulation, which is equivalent to the M-estimator of Huber&#x2019;s function solved via the iteratively reweighted least squares paradigm. Experimental results validate the effectiveness and robustness of the proposed algorithms.

Multi-Tone Phase Coding of Interaural Time Difference for Sound Source Localization With Spiking Neural Networks

Mammals exhibit remarkable capability of detecting and localizing sound sources in complex acoustic environments by using binaural cues in the spiking manner. Emulating the auditory process for sound source localization (SSL) by mammals, we propose a computational model for accurate and robust SSL under the neuromorphic spiking neural network (SNN) framework. The center of this model is a Multi-Tone Phase Coding (MTPC) scheme, which encodes the interaural time difference (ITD) between binaural pure tones into discriminative spike patterns that can be directly classified by SNNs. As such, SSL can be implemented as an event-driven task on highly efficient, neuromorphic parallel processors. We evaluate the proposed computational model on a directional audio dataset recorded from a microphone array in a realistic acoustic environment with background noise, obstruction, reflection, and other interferences. We report superior localization capability with a mean absolute error (MAE) of 1.02 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">°</sup> or 100% classification accuracy with an angle resolution of 5 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">°</sup> , which surpasses other SNN-based biologically plausible neuromorphic approaches by a relatively large margin and on par with human performance in similar tasks. This study opens up many application opportunities in human-robot interaction where energy efficiency is crucial. As a case study, we successfully deploy the proposed SSL system in a robotic platform to track the speaker and orient the robot's attention.

Efficient and robust approaches for three-dimensional sound source recognition and localization using humanoid robots sensor arrays

Efficient and robust sound source recognition and localization is one of the basic techniques for humanoid robots in terms of reaction to environments. Due to the fixed sensor arrays and limited computation resources in humanoid robots, there comes challenge for sound source recognition and localization. This article proposes a sound source recognition and localization framework to realize real-time and precise sound source recognition and localization system using humanoid robots’ sensor arrays. The type of the audio is recognized according to the cross-correlation function. And steered response power-phase transform function in discrete angle space is used to search the sound source direction. The sound source recognition and localization framework presents a new multi-robots collaboration system to get the precise three-dimensional sound source position and introduces a distance weighting revision way to optimize the localization performance. Additionally, the experiment results carried out on humanoid robot NAO demonstrate that the proposed approaches can recognize and localize the sound source efficiently and robustly.

Robust auditory localization using probabilistic inference and coherence-based weighting of interaural cues.

Robust sound source localization is performed by the human auditory system even in challenging acoustic conditions and in previously unencountered, complex scenarios. Here a computational binaural localization model is proposed that possesses mechanisms for handling of corrupted or unreliable localization cues and generalization across different acoustic situations. Central to the model is the use of interaural coherence, measured as interaural vector strength (IVS), to dynamically weight the importance of observed interaural phase (IPD) and level (ILD) differences in frequency bands up to 1.4 kHz. This is accomplished through formulation of a probabilistic model in which the ILD and IPD distributions pertaining to a specific source location are dependent on observed interaural coherence. Bayesian computation of the direction-of-arrival probability map naturally leads to coherence-weighted integration of location cues across frequency and time. Results confirm the model's validity through statistical analyses of interaural parameter values. Simulated localization experiments show that even data points with low reliability (i.e., low IVS) can be exploited to enhance localization performance. A temporal integration length of at least 200 ms is required to gain a benefit; this is in accordance with previous psychoacoustic findings on temporal integration of spatial cues in the human auditory system.

Fast Sound Source Localization Using Two-Level Search Space Clustering.

Steered response power phase transform (SRP-PHAT) is a method that is widely used for robust sound source localization (SSL). However, since SRP-PHAT searches over a large number of candidate locations, it is too slow to run in real-time for large-scale microphone array systems. In this paper, we propose a robust two-level search space clustering method to speed-up SRP-PHAT-based SSL. The proposed method divides the candidate locations of the sound source into a set of groups and finds a small number of groups that are likely to contain the maximum power location. By searching within the small number of groups, the computational costs are reduced by 61.8% compared to a previously proposed method without loss of accuracy.

A Modified SRP-PHAT Functional for Robust Real-Time Sound Source Localization With Scalable Spatial Sampling

The Steered Response Power - Phase Transform (SRP-PHAT) algorithm has been shown to be one of the most robust sound source localization approaches operating in noisy and reverberant environments. However, its practical implementation is usually based on a costly fine grid-search procedure, making the computational cost of the method a real issue. In this letter, we introduce an effective strategy that extends the conventional SRP-PHAT functional with the aim of considering the volume surrounding the discrete locations of the spatial grid. As a result, the modified functional performs a full exploration of the sampled space rather than computing the SRP at discrete spatial positions, increasing its robustness and allowing for a coarser spatial grid. To this end, the Generalized Cross-Correlation (GCC) function corresponding to each microphone pair must be properly accumulated according to the defined microphone setup. Experiments carried out under different acoustic conditions confirm the validity of the proposed approach.

Sound source localization for robot auditory systems

Sound source localization (SSL) is a major function of robot auditory systems for intelligent home robots. The steered response power-phase transform (SRP-PHAT) is a widely used method for robust SSL. However, it is too slow to run in real time, since SRP-PHAT searches a large number of candidate sound source locations. This paper proposes a search space clustering method designed to speed up the SRP-PHAT based sound source localization algorithm for intelligent home robots equipped with small scale microphone arrays. The proposed method reduces the number of candidate sound source locations by 30.6% and achieves 46.7% error reduction compared to conventional methods.

A Robust Sound Source Localization Approach for Microphone Array with Model Errors

In this paper, a robust sound source localization approach is proposed. The approach retains good performance even when model errors exist. Compared with previous work in this field, the contributions of this paper are as follows. First, an improved broad-band and near-field array model is proposed. It takes array gain, phase perturbations into account and is based on the actual positions of the elements. It can be used in arbitrary planar geometry arrays. Second, a subspace model errors estimation algorithm and a Weighted 2-Dimension Multiple Signal Classification (W2D-MUSIC) algorithm are proposed. The subspace model errors estimation algorithm estimates unknown parameters of the array model, i. e., gain, phase perturbations, and positions of the elements, with high accuracy. The performance of this algorithm is improved with the increasing of SNR or number of snapshots. The W2D-MUSIC algorithm based on the improved array model is implemented to locate sound sources. These two algorithms compose the robust sound source approach. The more accurate steering vectors can be provided for further processing such as adaptive beamforming algorithm. Numerical examples confirm effectiveness of this proposed approach.

System and process for robust sound source localization

System and process for robust sound source localization

System and process for robust sound source localization

A system and process for finding the location of a sound source using direct approaches having weighting factors that mitigate the effect of both correlated and reverberation noise is presented. When more than two microphones are used, the traditional time-delay-of-arrival (TDOA) based sound source localization (SSL) approach involves two steps. The first step computes TDOA for each microphone pair, and the second step combines these estimates. This two-step process discards relevant information in the first step, thus degrading the SSL accuracy and robustness. In the present invention, direct, one-step, approaches are employed. Namely, a one-step TDOA SSL approach and a steered beam (SB) SSL approach are employed. Each of these approaches provides an accuracy and robustness not available with the traditional two-step approaches.

Robust localization and tracking of simultaneous moving sound sources using beamforming and particle filtering

Mobile robots in real-life settings would benefit from being able to localize and track sound sources. Such a capability can help localizing a person or an interesting event in the environment, and also provides enhanced processing for other capabilities such as speech recognition. To give this capability to a robot, the challenge is not only to localize simultaneous sound sources, but to track them over time. In this paper we propose a robust sound source localization and tracking method using an array of eight microphones. The method is based on a frequency-domain implementation of a steered beamformer along with a particle filter-based tracking algorithm. Results show that a mobile robot can localize and track in real-time multiple moving sources of different types over a range of 7 m. These new capabilities allow a mobile robot to interact using more natural means with people in real-life settings.

Alpha-stable modeling of noise and robust time-delay estimation in the presence of impulsive noise

A new representation of audio noise signals is proposed, based on symmetric /spl alpha/-stable (S/spl alpha/S) distributions in order to better model the outliers that exist in real signals. This representation addresses a shortcoming of the Gaussian model, namely, the fact that it is not well suited for describing signals with impulsive behavior. The /spl alpha/-stable and Gaussian methods are used to model measured noise signals. It is demonstrated that the /spl alpha/-stable distribution, which has heavier tails than the Gaussian distribution, gives a much better approximation to real-world audio signals. The significance of these results is shown by considering the time delay estimation (TDE) problem for source localization in teleimmersion applications. In order to achieve robust sound source localization, a novel time delay estimation approach is proposed. It is based on fractional lower order statistics (FLOS), which mitigate the effects of heavy-tailed noise. An improvement in TDE performance is demonstrated using FLOS that is up to a factor of four better than what can be achieved with second-order statistics.